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The geometry of fundamental domains of lattices was used by Han and 
Wang to construct multivariate Gabor frames for separable lattices. We 
build upon their results to obtain Gabor frames with smooth and compactly 
supported window functions. For this purpose we study pairs of lattices 
which have equal density and allow for a common compact and star-shaped 
fundamental domain. The results are then extended to a larger class of 
i— i lattices via symplectic equivalence. 1 

O 1 Introduction 

Gabor systems are useful to construct discrete time-frequency representations of 
signals. A Gabor system is customarily denoted as (g, A) = {MJT x g : (x,u) G A}, 
where the square integrable function g is the Gabor window, A = MZ 2d , M 
full rank, a lattice in the time-frequency space IR 2d , T x the translation opera- 
tor (T x f)(t) = f(t - x), x G R d , and M w the modulation operator (M u f)(t) = 
e 2m ^^ f(t), to G ]R d . A Gabor system is a tight frame for the space of square in- 
tegrable functions L 2 (M. d ) if up to a scalar factor, Parseval's identity holds. That 



is, for some c > 0, we have 



f = c (f,M u T x g)M u T x g, f G L 2 (R d ), (1) 

where the series converges unconditionally and the computation of the represen- 
tation coefficients is stable. 



1 Math Subject Classifications: 42C15, 42C40 
Keywords and phrases: time-frequency lattices; lattice tilings and packings; symplectic equiva- 
lence. 
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Necessary and sufficient conditions for the Gabor system (g, A) to be a frame for 
L 2 (IR d ) have a long history in applied harmonic analysis. A classical statement is 
the density condition, which states that any Gabor system whose lattice A = MZ 2d 
has density d(A) = l/|detM| < 1 is not complete and therefore not a Gabor 
frame for L 2 (IR d ) [Dau90, Dau92, Jan94]. Recently Bekka showed that the density 
condition is sufficient as well: for each A with d(A) > 1 there exists a function 
g G L 2 (R d ) such that (<7,A) is a frame for L 2 (R d ) [Bek04]. In his fundamental 
paper, Bekka proves existence only and his work reveals nothing about the window 
besides membership in L 2 (M d ). Multivariate Gabor frames are constructed more 
explicitly for separable lattices in [HW01, HW04], however, the obtained Gabor 
windows are characteristic functions on sets that are fundamental domains for 
pairs of lattices. These fundamental domains may well be unbounded; in this 
case the constructed Gabor windows decay neither in time nor in frequency, so 
(1) is not local. For any lattice A, the existence of so-called multi-window Gabor 
frames with windows in the Schwarz space or in modulation spaces has been shown 
in [Rie88, Lue09], but the number of windows needed does not follow from their 
analysis, for example, it is not clear whether d(A) > 1 implies that a single window 
in the Schwarz space suffices. 

In this paper, we further explore the results of Han and Wang and use our 
findings to construct tight Gabor frames (g, A) in L 2 (IR 2 ) for separable lattices 
A = Ai x A 2 C M 4 with nonnegative, smooth and compactly supported windows, 
that is, g G C^°(1R 2 ). To achieve this, we shall assume star-shapedness of the 
common fundamental domain of a pair of lattices. Note that the so-called Balian- 
Low theorem [BHW98, GHHK02] implies that if (g, A) is a frame for L 2 (R d ) with 
g being in the Schwarz space <S(IR d ), then d(A) > 1. Hence, our analysis is limited 
to lattices with density greater than 1. 

The characterization of lattices A so that (g, A) is a frame for L 2 (R d ) for a fixed 
g are rare in the literature [Lyu92, SW92, JS02, Jan03], and with the exception 
of [GrolO, PR10] these concern the case d — 1 only. In principle, characterizing 
spanning properties of Gabor frames for lattices A of higher dimension (A C 
W. 2d , d > 2) is much more intricate than in the one-dimensional case. 

The paper is organized as follows. Section 2 recalls basic facts from Gabor 
analysis and from the Fourier theory of translational tilings that are used in this 
paper. Section 3 contains results from [HW01, HW04] as well as minor extensions 
of these. The construction of smooth and compactly supported Gabor windows for 
a class of separable lattices is presented in Section 4. These results extend to some 
lower block-triangular matrices, Section 5 contains examples in the bivariate case, 
that is, we present pairs of lattices in IR 4 which allow for a common star-shaped 
fundamental domain. 
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2 Tools in time-frequency analysis 



Throughout the text, all subsets of R d are assumed to be Lebesgue measurable. 
For fl C R d , we use fl + x = {y + x : ye fl}, x G R d , ~/fl = {73/ : y G fl}, 7 > 0, 
and fl e = {y + z : y e fl, \\z\\2 < e}, e > 0, where || H2 denotes the Euclidean 
norm on M d . The characteristic function on fl C R d is denoted by xn, that is, we 
have xq(^) = 1 if ^ G ^ and — e l se - 

We use the standard notation for complex- valued functions on R d : «S(R d ) is 
the Schwarz space, and C£°(R d ) the space of smooth and compactly supported 
functions. 

A time-frequency shift is (n{X)f)(t) = (M u T x f)(t) = f(t - x)e 2 ^^ for A = 
(x,oj) G WL 2d . The Fourier transformation used here is L 2 (M d )-normalized, that is, 
/(0 = Imd f(y)e' 2ni ^ y dy for / integrable. 

Definition 2.1 A lattice A in M. 2d is a discrete subgroup of the additive group M. 2d , 
that is A = MZ d , with M being full-rank (detM ^ 0). A separable lattice has the 
form A = AZ d xBZ d . The dual lattice of A = MZ d is A 1 - = M~ T Z 2d , and the ad- 
joint lattice of A = MZ 2d is A° = {A G R d x R d : 7r(A)7r(/i) = for all (i G 
A}. The volume of the lattice A = M7L d equals the Lebesgue measure ofR d /A, 
that is vol A = m(R d /A) = |detM] ; and the density of A is d(A) = (volA) -1 . 

We have g^A -1 ) = d(A°) = l/d(A), and the adjoint A° of a separable lattice 
A = AZ d xBZ d is A° = (BZ d ) ± x(AZ d ) 1 - = (B- T Z d )x(A- T Z d ), where M~ T 
denotes (M T )-\ Moreover, (M°)° = M and (M ± ) ± = M [GroOl]. 

Definition 2.2 Let Q C R d be measurable and let A be a full rank lattice. If 
(Q + Ai) fl (Q + A2) is a null set for Ai 7^ A2, Ai, A2 G A ; then Q is a packing set for 
A. If in addition, R d = U\ e A(Q + A), then Q is a tiling set (fundamental domain) 
for A. 

Clearly, a measurable set Q is a packing set if and only if XIagA Xn{x— A) < 1 for 
almost all i6l' ! and a fundamental domain for A if and only if XaeA Xn( x ~ = 
1 for almost all x G M d . Furthermore, if SI is a packing set and m(Q) = vol A, 
then fl is a fundamental domain for A. Moreover, any translate of a fundamental 
domain of A is a fundamental domain of A. 

If Q is a star-shaped fundamental domain for A, that is, there exists a point 



N G fl such that for all Q G fl the line segment NQ is contained entirely within 
fl (see Figure 1), then j(fl — Q) is a packing set for A for all < 7 < 1. 

Fundamental domains for lattices can be characterized with methods from 
Fourier analysis [Kol04]. 

Lemma 2.3 Let A be a lattice in R d and fl C R d be measurable, fl is a funda- 
mental domain for A if and only if Xn vanishes on A^jO}. 

Moreover, the following result is central to our analysis [Fug 74, IKT03, KM06]. 
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Theorem 2.4 The set Q is a fundamental domain of the lattice A in M. d if and 
only if the set of pure frequencies {e 2mXx }xeA ± ^ s an orthonormal basis for L 2 (Q). 

We recall some important definitions and properties from the theory of Gabor 
frames and Gabor Riesz basis sequences. 

Definition 2.5 A Gabor system (g,A) is a frame for L 2 (M. d ) with frame bounds 
< a < b if 

a\\f\\h < E Ki>(%>| 2 < h Wf\\h for all f G L 2 (R d ). (2) 
AeA 

The Gabor frame is tight if a = b is possible. A Gabor system (g, A) is a Riesz 
basis for L 2 (M. d ) if there exist constants < a < b such that 



1 1 2 

a||c||| 2 < Vc a tt(A)# <6||c||| 2 for all c G £ 2 {A), (3) 

1 1 ' L 2 

AeA 



and span (g, A) = L 2 (M. d ). A Gabor Riesz basis sequence is a Riesz basis for the 
L 2 -closure of its linear span. 

Clearly, if (g, A) is a Riesz basis sequence with ||g||z,2 = 1 and a = b = 1 in (3) then 
(g, A) is is an orthonormal sequence. Moreover, we shall use below that if (g, A) 
is a tight Gabor frame, then a = b = d(A)||^||^ 2 in (2) and c = (d(A) HpH^) -1 in 
(1) (Theorem 5 in [BCHL06]). 

The usefulness of Gabor frames and tight Gabor frames stems from the follow- 
ing result. 

Theorem 2.6 Let g G L 2 (M. d ) and let A be a full rank lattice. If (g, A) is a frame 
for L 2 (M. d ), then exists a so-called dual window 7 G L 2 (M. d ) with 

f = J2(f^( x hM x )9 = J2^^ x )9)^H feL 2 (R d ). (4) 

AeA AeA 

If(g,A) is a tight frame for L 2 (WL d ) , then we can choose in (4) 
7 = (d(A)||#||^ 2 ) -1 g, that is, (1) holds with c — (d(A)\\g\\ 2 L2 )~ 1 . 

The benefit of having compactly supported and smooth g in (1) and (4) is clear. 
Only then, we can guarantee that for any r G N and any compactly supported 
r-times different iable function /, there exists Nf,Cf > with (f,ir(x,u)g) = 
whenever ||x||2 > Nj, and |(/, 7r(x,a;)^)| < C|w|~ r . This property also allows for 
efficient quantization of the expansion coefficients in (1) and (4) [Yil03]. 

For a detailed discussion of Bessel sequences, Gabor frames and Riesz basis 
sequences, we refer to [GroOl, Chr03]. 

The following results are central in the theory of Gabor frames [RS97, FG97, 
FZ98]. 
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Theorem 2.7 Let g G L 2 (R d ) and let A be a full rank lattice. Then (<?, A) is a 
frame for L 2 (R d ) if and only if (g,A°) is a Riesz sequence. Moreover, (g, A) is a 
tight frame for L 2 (R d ) if and only if (g, A°) is an orthonormal sequence. 

Gabor frame theory is rooted in the representation theory of the Weyl-Heisenberg 
group, a fact that we shall exploit in Sections 3 and 4 [Fol89, GroOl]. In particular, 
we shall use so-called metaplectic operators which are discussed below. 

Definition 2.8 The symplectic group Sp(rf) is the subgroup of GL(2d,WL) whose 
members (db) are characterized by AD T = A T D, BC T = B T C and A T B — 
D T C = I. A lattice A = MZ 2d with M G Sp(rf) is called symplectic lattice. 

Theorem 2.9 For M G Sp(rf) exists a unitary operator /i(M) on L 2 (R d ), a so- 
called metaplectic operator, with -k(M\) = /i(M )*7r(A)/i(M) ; A G R 2d . 

Theorem 2.10 Let A be a full rank lattice in WL 2d and M be a symplectic matrix 
in GL(WL,2d). Then the following are equivalent: 

1. There exists g G L 2 (R d ), respectively g G 5(lR d ) 7 such that (g,A) is a Gabor 
frame for L 2 (R d ). 

2. There exists g G L 2 (R d ), respectively g G 5(lR d ) ; such that (g,MA) is a 
Gabor frame for L 2 (R d ). 

Theorem 2.10 follows from Theorem 2.9 and the choice g = fi(M)g. In fact, all 
metaplectic operators restrict to <S(R d ) [GroOl], but unfortunately not to C£°(R d ) 
as discussed in the next paragraph. In general, the spanning properties of the 
Gabor system (g, A) are transferred to the Gabor system (fi(M)g,MA) [GroOl]. 
Thus, we can replace the quantifier 'a Gabor frame for L 2 (R d )' by 'Riesz basis 
sequence' or 'frame sequence'. 

Note that Sp(oQ is generated by matrices of the form ( -?/ o ) > ( o b°- t ) > ^ & 
0, and C - symmetric. The corresponding metaplectic operators are, re- 

spectively, the Fourier transform, the normalized dilation / i— >■ | &etB\ l l d f o B, 
and the multiplication with a chirp e nt( - x ' Cx K Clearly, /i(M) restricts to C^°(lR <i ) 
if M is generated by matrices of the form b -t), and °), C - symmetric, a 
fact that we shall exploit below. 

3 Characteristic functions as Gabor frame win- 
dows 

Han and Wang construct Gabor systems with windows that are characteristic 
functions on fundamental domains of pairs of lattices [HW01, HW04]. Their con- 
struction is based on part 1 implies part 3 of the following result. 
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Proposition 3.1 Let Q, C R d be a fundamental domain for AZ d . The following 
are equivalent. 

1. Q is a packing set for B~ Tr L d ; 

2. (xn, AZ d x BZ d ) is a frame for L 2 (R d ); 

3. (xn, AZ d x BZ d ) is a tight frame for L 2 (R d ) with frame bound 1/| det B\; 
4- (xn, B~ T Z d x A~ T Z d ) is a Riesz basis sequence; 

5. ((m(r2) -1 / 2 xn, B~ T Z d x A~ T Z d ) is an orthonormal sequence. 

Note that Q being a fundamental domain for AZ d and a packing set for B~ T Z d 
implies | det-B~ T | > | det A\ and for A = AZ d x BZ d , we then have d(A) > 1 and 
for A = B~ T Z d x A~ T Z d , we have d(A) < 1. 

Proof We shall show that part 1 implies part 5, and part 4 implies part 1. 
Clearly, part 3 implies part 2 and part 5 implies part 4. The equivalence of parts 
2 and 4 and parts 3 and 5 follows from Theorem 2.7. The explicit frame bound 
in part 3 follows from \\xn\\ 2 L 2 = | det A| and d(AZ d x BZ d ) = l/\detA detB\ 
[BCHL06]. 

Note that Q being a packing set for B~ T Z d implies 

(n(x, u)xn, 7r(x', w')xn> = 0, (x, u), (x' , u') G B~ T Z d x A~ T Z d , with x^x . 

Moreover, the family {M^Xn+x '■ w G A~ T Z n } is an orthogonal basis for L 2 (f2+x), 
x G R. d , by Theorem 2.4, so part 4 follows. 

Now, if Q is not a packing set for B~ T Z d , then there exists x G B~ T Z d , 
W = Q, n (n + x) with m(W) > 0. Note that W - x = (ft - x) n ft C fi. As ft is 
a fundamental domain for AZ d , there exists {c L0 } UJ (zA- T z d with 

Xw + Xw-x = C <^ M ^ • 

u<=A- T Z d 

For iVeN, consider 

f» = E E (- l ) fec ^M u;Xn 

fc=i weA- T z d 

JV 

= E^ _ -O^XW+fcz + XW+(fc-l)x) — { — ]) N XW+Nx ~ XW ■ 
k=l 

Clearly, ||/at|||2 < 2m(W), but the coefficients {c4,w}(>,w)eB- T z d xA- T z d °f t ne Ga- 
bor expansion of f N in terms of [xci,B~ T Z d x A~ T Z d ) satisfy 

N 

E kj 2 = E E i^i 2 = Ar E kj 2 >ivm(^). 

(a;,w)eJ3- T Z d xA- T Z d fc=l ui€A- T Z d io&A~ T Jj d 
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This implies that (xn, B T Z d x A T Z d ) is not a Riesz basis sequence. 

□ 

Han and Wang's central result is the following. 

Theorem 3.2 Let AZ d and BZ d be two full-rank lattices in R d , such that | det A\ = 
| det.B|. Then there exists a measurable set Q which is a fundamental domain for 
both AZ d and BZ d . If |det^4| > |detS| ; then there exists a measurable set Q, 
which is a packing set for AZ d and a tiling set for BZ d . 

The combination of Proposition 3.1 and Theorem 3.2 provides us with the main 
result from [HW01, HW04]. The results were the first to prove the existence of 
Gabor frame windows for any separable lattice with density greater than or equal 
1. 

Theorem 3.3 

1. If | det A det B\ = I, then there exists ttCR d such that (xn,AZ d x BZ d ) is 
an orthonormal basis for L 2 (M. d ). 

2. If | det A det 5 1 > I, then there exists C ]R d such that (xn, AZ d x BZ d ) is 
an orthogonal sequence for L 2 (R d ). 

3. If | det A det 5 1 < I, then there exists C ]R d such that (xn, AZ d x BZ d ) is 
a tight Gabor frame for L 2 (M. d ) with frame bound 1/| det_B|. 

Proof. Parts 2 and 3 follow directly from Theorem 3.2 and Lemma 3.1. Part 
1 follows from part 2 and the observation that Q tiles with respect to AZ d and 
B- T Z d , then (xn, AZ d x BZ d ) is complete. □ 

Proposition 3.1 states that if fl C M. d is a fundamental domain for AZ d , re- 
spectively B~ T Z d , then f2 must be a packing set for B~ T Z d , respectively AZ d , in 
order for (xn, AZ d x BZ d ) to be a frame, respectively a Riesz basis sequence. The 
interplay of the fundamental domain property of one lattice with the packing set 
property with respect to a second lattice is further illuminated by the following 
observation. 

Proposition 3.4 Let Vt C R d . 

1. If (xn,AZ d x BZ d ) is a frame for L 2 (M. d ), then Q contains a fundamental 
domain of AZ d . 

2. If Vl is a packing set for B~ T Z d , then (xn, AZ d x BZ d ) is a frame for L 2 (IR d ) 
if and only if Q contains a fundamental domain of AZ d . 

3. If Q is a packing set for B~ T Z d , then (xn,AZ d x BZ d ) is a tight frame for 
L 2 (R d ) if and only if Q is the union o/fceff disjoint fundamental domains 
of AZ d . The frame bound is then k/ \ det£>|. 
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4- If (xn, AZ d x B7L d ) is a Riesz basis sequence, then fl contains a fundamental 
domain of B~ T Z d . 

5. If fl is a packing set for AZ d , then (xn, AZ d x BZ d ) is a Riesz basis sequence 
if and only if fl contains a fundamental domain of B~ T Z d . 

6. Ifflisa packing set for AZ d , then (xn, AZ d x BZ d ) is an orthogonal sequence 
if and only if fl is the union of a finite number of fundamental domains of 
B- T Z d . 



Proof. Theorem 2.7 implies that it suffices to show parts 1, 5, and 6. The first 
statement is trivial, as else (xn, AZ d x BZ d ) would not span L 2 {R d ). 

In the remaining parts, we assume that fl C R d is a packing set for AZ d , hence, 
it suffices to show that {M^xn}ujeBz d is a Riesz basis sequence if and only if fl 
contains a fundamental domain of B~ T 7^ d , respectively an orthogonal sequence if 
and only if fl is the union of a finite number of fundamental domains of B~ T Z d . 

If fl contains a fundamental domain of B~ T Z d , then clearly {M u ,xci}u,eBz d is 
a Riesz basis sequence [Chr03]. If fl does not contain a fundamental domain of 
B- T Z d , then J2^b-t^ Xn+u = on a set W C B- T [0, l) d with m(W) > 0. Note 
that we have also XLe£- T z d Xsi+u> = on a set + re for any x e B~ T Z d . 

As B~ T [0, l) d is a fundamental domain for B~ T 7^ d , there exist {c 7 } 7eBZ d ^ 
such that xw = Y. c -y M iXB-T[o,i) d - Then ^ c 7 M 7 x R d = ^2 ueB -r Z d Xw+u and 

° = Xn{ Yl Xw+oj) = Yl c ~f M iXn- 

weB- T Z d -yeBZ d 

We conclude that {M co xa} w( zBz d is n °t a Riesz basis. 

Now, if fl is the finite union of fundamental domains of B~ T Z d , then clearly 
{M UJ xn}ueBz d is an orthogonal sequence. On the other side, if {M u! xn} U j£Bz d is 
an orthogonal sequence, then for oj ^ 0, 

= / M wX n{t)dt= [ e 2muJt Xn (t)dt 



- £ f 

= [ V e 2 ^ t - p hn(t-p)dt 

JB-T [0 ,l) d peB _ Tlid 

-L 



Xn(t-p)dt. (5) 

■-B- T 1 d 

Hence, ^2 pe B- T z d Xn(t — p) is an integer- valued constant function, that is, 
Y P <=B- T z d Xn(t—p) = k G N. This implies that almost every point in R d is covered 
by k translates of fl. Hence, fl is the union of k fundamental domains. In fact, 
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we can find a fundamental domain Qi C f2, remove it as shown below, and then 
continue inductively. 

As long as J2 peB -T Z d Xn(t — p) > 1 on a set of nonzero measure, there exists 
q G B~ T Z d such that f2 + q fl f2 has nonzero measure. Set ft' — Q \ (Jl + q). Now, 

U («+p)= U + 

P eB- T z d P eB- T z d 

follows from IJ^ez + = U^ez + > which i n turn follows from f2 C 
U £ez (fi' + ^g). But if x G n \ (n' + iqfj, then x G ft + £q for all £ G N, a 

contradiction. □ 

Remark 3.5 For (xa, AZ d x i?Z d ) to be a frame for L 2 (IR d ) it is not necessary 
that C R d is a packing set for B~ T Z d . In fact, (x[o,3/2), 1/2Z x Z) is a frame 
[Jan03]. 

We close with a simple, but interesting example. 

Example 3.6 Let W be a fundamental domain for 1/2Z x Z, (x, y) G M 2 , and 
fl = W + (x, y) U W. Then (xn, 1/2Z x Z x Z x Z) is a frame for L 2 (M 2 ) if fl is a 
packing set for (Z x Z)~ T = Z x Z. A simple computation shows that this is the 
case if and only if W fl W ± [k + x, £ + y) = for all (k, £) G Z 2 . If Q is a packing 
set for Z x Z, then (xn, |Z x Z x Z x Z) is a tight frame for L 2 (IR 2 ) if and only 
if x G (2Z + 1) and y G Z. For a detailed discussion see [PR10] 

4 Existence of smooth and compactly supported 
Gabor frame windows 

Han and Wang's construction of Gabor orthonormal bases for separable lattices is 
a landmark result in Gabor analysis. The drawback of their approach lies in the 
fact that the constructed functions are discontinuous and may not even decay at 
infinity. In this section, we will construct nonnegative, compactly supported, and 
smooth window functions for a class of separable lattices. 

We begin this section with a simple result to indicates a limitation of Theo- 
rem 3.3 as well as the direction which we will take. Its derivation is included in 
the appendix. 

Proposition 4.1 Let A = AZ d x BZ d with d(A) > 1. Let Q be a fundamental 
domain for AZ d and a packing set for B~ T Z d . If g G C(lR d ) and supp g = Q, then 
the Gabor system (g,A) is complete, but not a frame for L 2 (M. d ). 

In view of Proposition 4.1 we have to consider window functions in C£°(R d ) 
whose support extends beyond the fundamental domain Q of A7* d . Recall that 
Q € = {x + y, x G f2, \\yW2 < e}- Further, G C^°(lR <i ) is nonnegative and satisfies 
supp0 C -Bi(O) = {1 G l d : ||x|| 2 < 1} and J <fi = 1. Further e (x) = l/e<f)(x/e). 
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Theorem 4.2 



1. If there exists Q C W 1 , e > ; such that Q is a fundamental domain for AZ d 
and fL is a packing set for B~ T Z d , then {^/xn * (fie, AZ d x BZ d ) is a tight 
frame for L 2 (M. d ) with frame bound 1/| det B\. 

2. Suppose there exists Q C ]R d ; e > ; such that Q is a fundamental domain 
for B~ T Z d and Q t is a packing set for AZ d , then (y/x~ci * (fie, AZ d x BZ d ) is 
an orthonormal system. 

Note that the conditions on Q imply d(A) < 1 in the first statement and 
d(A) > 1 in the second statement of Theorem 4.2. 

Proof. We shall prove the second statement, the first statement follows then 
from Theorem 2.7 and the observation that 

II V Xn * <Pe\\ 2 L 2 = J \xn*<Pe\ = J Xn = m(ti) = | det A\. 

Following the proof of Theorem 3.3, we obtain that (xn, A) is an orthonormal 
system. Moreover, suppxn * 0e Q ^e, and as Q t is a packing set for AZ d , we 
maintain n(k, £)y/xn* (fie is orthogonal to ir(k f ,£ f )xn * (fie if k ^ k' . It remains to 
show that {7r(0, * (fie} is orthogonal. But this follows as for £ ^ £' 

(A0J)Vx^e , A0J')Vx^e) = J e 2 ^ e ~ e >xn * (fie{x) dx 

= (xn * <fieT{£' -£) = xn(t' - £)&(£' -£) = o. 

□ 

To construct sets that allow for the application of Theorem 4.2, we turn to 
lattices which have starshaped common fundamental domains. 

Proposition 4.3 // < | det A| < |det£| and \ det B / det A\ 1/d AZ d and BZ d 
have a bounded and star-shaped common fundamental domain, then exists Q C M. d 
such that fl e is a packing set for BZ d and Q is a tiling set for AZ d . 

Proof. We let A = | det B / det A\ 1/d AZ d , and obtain | det Aj = | det Sj. By 
Theorem 3.2 there exists a measurable set Q' which is a common fundamental 
domain for AZ d and BZ d , and by hypothesis, we can assume Q' is star-shaped. 
We claim that there exists a fundamental domain Q for AZ d such that Q e C Q'. 
By o ur hypothesis, we choose a point N E £1' such that for all Pefl', the segment 
is contained in the interior of Q'. Without loss of generality we may take N 
to be the origin (Figure 1). We apply a dilation with center N and coefficient 
| det 5/ det A\~ 1/d to Q' and obtain a set Q with Q n Q' = Q. Clearly, Q is a 
fundamental domain for the lattice AZ d . □ 
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Figure 1: SI is the scaled copy of fl' under dilation with center N. 

Corollary 4.4 For A = AZ d x BZ d with d(A) > 1 suppose that the lattices 
d(A) 1 ^ d AZ d and B~ T Z d have a bounded and star-shaped common fundamental do- 
main Q, then exists a nonnegative g G C^°(IR <i ) with (g,A) being a tight frame for 
L 2 (R d ). 

Proof. The result follows from Theorem 4.2, part 2, and Proposition 4.3 where 
we replace B by B~ T and note that 



1 < d(A) = 



1 




det B~ T 


det A det B 




det A 



□ 

Corollary 4.4 can be extended to a class of upper and lower-block triangular 
lattices. 

Corollary 4.5 // the lattices AZ d and | det A det B\ l/d B~ T Z d have a bounded 
and star-shaped common fundamental domain Q' and if D is such that DA^ 1 
symmetric, then exists g e C^(R d ) such that {g,{^%) Z 2d ) is a frame for L 2 (R d ). 

Proof. Let T = (.^J). Then T(^%)Z d = AZ d x BZ d is separable 
and fulfills the conditions of Corollary 4.4, so there exists g G C^°(lR d ) such that 
(g, TA) is a frame for L 2 (M. d ). Since T is symplectic, by Theorem 2.10 there exists 
g G S(R d ) such that (g, A) is a frame for L 2 (R d ). Furthermore, g G C c °°(M d ) 
because the metaplectic operator associated to T is a multiplication by a chirp, 
which preserves the compact support of g [Fol89]. □ 

5 Bivariate examples 

In this section we provide several examples which illustrate the geometric criteria 
established above for a family of matrices in the case d = 2. 
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Figure 2: The set f2 constructed in Proposition 5.1. 

Proposition 5.1 Let q G Q + and m,n co-prime integers such that q = m/n. 
There exists a common convex fundamental domain for Z 2 , y q ^j Z 2 and ( 1 /™ ™) Z 2 . 

Similarly, the lattices Z 2 , ^™d m m / n j Z 2 and i/n) ^ 0,ue fl common convex 
fundamental domain. 

Proof. We shall only prove the first assertion, the second one follows analo- 
gously. 

Let Q — ( 1 / n ™) [0, l) 2 . f2 is a convex set and m(f2) = 1. Suppose that there 

exists (k, l) T ^ if G Z 2 such that {fi + (k, l) T } n f2 7^ 0. Then there exist points 
(a u l3 1 ), (a 2 ,l3 2 ) G [0, l) 2 such that f + mfc + k = f + m/3 2 and nfc + l = n/3 2 . 
Therefore, (3 2 — (3\ = -, which implies that ol\ — a 2 = ml — kn G Z. Since < 
ai,a 2 < 1, necessarily ol\ = a 2 , and also that (3 2 — (3i = —. Since gcd(m,n) = 1 
and0</3 2 -/3i < 1, this is possible only if k = I = 0. Thus^ + Z^O}) Dtt = 0. 
Since m(Q) — 1, is a fundamental domain for Z 2 . 

A similar proof shows that Q is also a fundamental domain for the lattice 

( ml : 1 ° )z 2 . □ 

V n/rn J 

To illustrate strength and weakness of our method, we shall consider the fol- 
lowing, apparently simple example. 
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Corollary 5.2 If A = aZ x bZ x cZ x dZ with d(A) = abed < 1 satisfies 




(1) ac < l,bd < 1, or (2) a&cd < 1/2, or 

£/ien there exists g E C^°(IR 2 ) sfic/i £/iai (g, A) zs a tag/it frame for L 2 (R 2 ). 

Proof. (1) If ac < 1 and fee? < 1, then any g 1 E C£°(R) with X[o,a] < 9i < 

X[(oc-l)/(2c),(oc+l)/(2c)] an d °2 £ witn X[0,6] < 92 < X[(bd-l)/(2d),(bd+l)/(2d)] 

guarantees that {g\,aZ x cZ) and (g 2 ,bZ x dZ) are frames. A simple tensor 
argument then implies that (gi <S> 92, aZ x &Z x cZ x dZ) is a frame for L 2 (IR d ). 
Note that the same line of argument shows that if either ac > 1 or bd > 1, then 
exists no g 1 , g 2 E C£°(R) with (g 1 <g> g 2 , aZ x 6Z x cZ x dZ) is a frame for L 2 (M d ) 
[PR10]. 

(2) It suffices to consider abed < 1/2, and ac > 1 or bd > 1, as else, (1) would apply. 
Without loss of generality, we consider ac > 1, and, hence bd < 1/2. Moreover, 
applying Theorem 2.10 with the symplectic matrix M = diag(c, d, 1/c, 1/d) implies 
that the existence of g E C^°(]R 2 ) with (g, A) being a frame for L 2 (IR 2 ) follows from 
the respective statement for A' = diag(c, d, 1/c, 1/d) A = diag(ac, bd, 1, 1)Z 4 . With 
= ( 1/2 bd) [0, l) 2 , e = any g G C^°(]R 2 ) with 

Xn < 9 < Xn. 
has the property that (g, A') is a frame for L 2 (IR 2 ). 

(3) Theorem 5.4 applies whenever there exist m,n E Z, a E R such that am 2 = ac 
and an 2 = bd, which is equivalent to G Q. □ 

The conditions on A presented in Proposition 5.2 are not necessary for the 
existence of g G C^°(IR 2 ) with (g,A) being a Gabor frame for L 2 (IR d ). In fact, 
if for g E C^°(]R 2 ), (g, MZ 4 ) is a Gabor frame for L 2 (IR 2 ), then exists an open 
neighborhood U of M in GL(R 4 ), such that (g, M'Z 4 ) is a Gabor frame for L 2 (M d ) 
for all M' E U [FK04] . For results where rationality of lattices plays a central role, 
see results known as Janssen's tie [Jan03]. 

Below, we show that the condition abed < 1 and the use of diagonal matrices 
with rational entries in Corollary 5.2 is critical for our method to be applicable. 
This clearly illustrates the limitations of the method described in Theorem 3.3. 

Proposition 5.3 There exists no fundamental domain Q for Z x |Z with Q e , 
e > 0, is a packing set for Z x Z. Consequently, there exists no common star- 
shaped fundamental domain for Z 2 and (^j_jZ 2 . 



V2 

Proof. Suppose that f2 is a tiling set for Z x |Z. If il e is a packing set for 
Z x Z, then {fl + (m, n) : m, n E Z} have no boundary points in common. Hence 
all sets {Q + (m, |) : m, n E Z} with common boundary point with Q must be of 
the form Q + (m, n + |). Clearly, there must be two such sets in the corona of Q 
which have a common boundary points. But this is a contradiction as the system 
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{Q + (m, n + |) : m, n G Z} is a translate of the system {f2 + (m, n) : m,n E Z} 
and hence all of its members should have disjoint boundaries. 

To obtain the second assertion, assume that there exists a compact star-shaped 
set Q' serving as a common fundamental domain for both lattices. Then there 
exists x G IR 2 ,e > such that for Q = Dj_Q' + x, we have Q C Q e C Q'. 

Note that is a tiling set for ^ ^ ^ ^ j Z 2 = (o § ) ^ 2 > contradicting the first 
assertion. 

□ 



Theorem 5.4 Let m,n G Z &e relatively prime. Let A = AZ 2 x 5Z 2 fre a lattice 
in M 4 . Whenever B T A is of the form 

1. al , \a\ < 1; 

^' ("o"^); where \a\ < (mn)^ 1 ; 
3. (;X), where \a\ < n~ l ; or 

I {£n a aa)> where H < ""S 



t/ien i/iere exists a function g G C, 



c°°( 



2 ) snc/i £/ia£ (<7,A) a Gabor frame for 



Proof. We have 



A 
£ 



B- T 0\ AB T A 



B 







M 



This shows that A = M((B T A)l? x Z 2 ) with M symplectic. Since | det B T A\ < 



1, we can rescale £? T AZ 2 to make its density I. Then |det-B T A| 
respectively of the form 



l / 2 B T AZ 2 



is 



Z 2 



m/n 
n/m 



Z 2 



1/n 




m 
n 



Z 2 



rn 





1/n 



Z 2 



Proposition 5.1 assures the existence of a common convex fundamental domain 
for (B T A)Z 2 and Z 2 accordingly. By Theorem 3.3 there exists g' G C7?°(1R 2 ) such 
that ((?', (B T A)Z 2 x Z 2 ) is a Gabor frame for L 2 (IR 2 ). The matrix M is symplectic, 
and its associated metaplectic operator /x(M) from Theorem 2.10 is the dilation 
(/i(M)h)(x) = \ det B\- ^ hiB^x), [Fol89]. Hence, g = G C C °°(M 2 ) and 



((?, A) is a Gabor frame for L 2 (R 2 ). 

Note that it is not known for which a, (5 there exists g G 



□ 

s2 ) such that 



(g,Z 2 x (op) is Gabor frame for L 2 (R d ). If #o(^) = e ^^l' 2 is a Gaussian, then 
(^ ,Z 2 x (g J) is a frame for L 2 (M d ) if a J < 1 [PR10]. 
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6 Appendix: Proof of Proposition 4.1 

The operator 

S g , A f = Tr(X)g) n(\)g, f G L 2 (R d ) 

AeA 

is called a Gabor frame operator. It is positive and self-adjoint if (g, A) is a frame 
for L 2 (R d ) [GroOl, Chr03]. Gabor frames possess a very useful reconstruction 
formula: 

/ = £</, *(A)ff> vr(A) 7 = 7r(A) 7 ) tt(%, / G L 2 (R d ) 

AeA AeA 

with 7 = S~ A g being the so-called canonical dual window [GroOl, Chr03]. 

S g ,Az d xBz d can be represented in matrix form [Wal92, RS97]. For that purpose, 
we define the bi-infinite cross-ambiguity Gramian matrix 

G(x) = {Gjk{x))j^ keIj d : 

G jk (x) = \detB\- l Y^ g(x - B~ T k - A£)g(x - B~ T j - Ai). ( 6 ) 

Below, VF(IR d ) denotes the Wiener space, consisting of all functions such that 
the norm 

\\f\\w = ii/ • 7fcX[o,i) d iioo 

is finite [Chr03]. 

Proposition 6.1 Let g G W(R d ). Let A = AZ d x BZ d be a full-rank lattice in 
R 2d . For f,h G L 2 (R d ), define the sequences 

f(x) := {f(x - B~ T j) : j G Z d }, h(x) := {h(x - B~ T k) : k G Z d }. 

Then the following holds: 

(S gA f,h)= [ (G(x)f(x)Mx))dx 

JB- T [G,l) d 

for all f,h G L 2 (R d ). S 9: \ is a bounded operator on L 2 (R d ) if and only if there 
exists b > such that G(x) < hip. for almost all x G R d . Also S 9j \ is a boundedly 
invertible operator on L 2 (R d ) if and only if there exists a > such that G(x) > alp 
for almost all x G R d . 

Note that {g, A) is a Gabor frame for L 2 (R d ) if and only if S 9i a is bounded and 
boundedly invertible on L 2 (R d ). 

Proof of Proposition 4-1- Let Q be a fundamental domain for A7* d and packing 
set for B' T Z d , g G C(R d ) with supp g = ft. Let / G L 2 (R d ). Denote by f k 
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the restriction of / to Q + Ak, k G Z d . Thus ||/||| = ^ k&L d where f k G 

/. 2 (<) • .1/.-). 

To prove completeness, suppose there exists / G L 2 (R d ) such that 

(f,M Bl T Ak g) = 0, k,leZ d . (7) 

However, because suppg = Q, for a fixed A; G Z d , (7) is the Fourier coefficient 
(fk ' TAkgY~{Bl). From the Fourier series expansion 

(f k ■ T Ak g)(t) = T^gTiBt) e™ 

we see that • T^g is identically almost everywhere on f2 + A/c. Because g 
does not vanish on a subset of f2 of positive measure, f k — almost everywhere 
on f2 + AA; for all fc. Therefore / = almost everywhere on IR d and completeness 
of (g>, A) is shown. 

Assume now that (g, AZ d x BZ d ) is a Gabor frame for L 2 (R. d ). We analyze the 
structure of the associated cross-ambiguity matrix G(x). If j ^ k, then 

supp g(x - B- T k - Al)g{x - B~ T j - Al) C Al + [(ft + 5~ T A;) n (ft + S _T j)]. 

Since m((ft + B~ T k) H (ft + B~ T j)) = 0, for almost all a; we have G k j(x) = for 
j 7^ A;. Thus the matrix G(x) given by (6) is diagonal for almost all x. Moreover, 

G 0O (x) = y £\g(x-Al)\ 2 = \g(x)\ 2 , 

since suppg = ft. When c = {So(n)} ne z d , Proposition 6.1 implies that a < 
(G(x)c,c) <b,b ecause S g is a bounded and invertible operator on L 2 (M. d ). There- 
fore, the associated matrix G(x) shares these properties for almost every x. But 
then (G(x)c, c) = G 00 (x) = \g(x)\ 2 , which in turn implies a < \g(x)\ 2 < b on ft, 
contradicting the continuity of g on M. d . □ 
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